MOTION PICTURE ANALYZING SYSTEM 



BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates to a motion picture 
analyzing system that analyzes a motion picture such as 
a video image, and more specifically, relates to a 
motion picture analyzing system, a motion picture 
analyzing method and a motion picture analyzing program, 
all of which analyze the motion of an object in a motion 
picture . 

Description of Related Art 

Generally, a motion picture such as a video image 
consists of a group of still pictures photographed at a 
discrete time interval (photographing interval) . 
Conventionally, when the motion of an object in a motion 
picture is analyzed, a method, in which a pixel having a 
temporal change is extracted by comparing images of 
still pictures photographed at different discrete time 
points, is generally adopted. 

With such a conventional method, however, 
continuous motion of an object is caught at a discrete 
photographing interval; thus, as the motion of the 
object becomes faster, it becomes difficult to find 
corresponding pixels by temporally comparing the images 
of the consecutive still pictures, and it also becomes 
difficult to analyze the motion of the object in a 
motion picture. 

As a method to solve such a problem, there have 
been heretofore proposed: a method (1) for shortening 
the photographing interval to such a degree that fast 
movement of the object can be caught; and a method (2) 
for interpolating data between images of still pictures 
as temporally consecutive discrete data by software. 

However, with the method (1), since it is necessary 
to reexamine the configuration of the photographing 



equipment basically to shorten the photographing 
interval, it may not be practical in view of the cost. 
With the method (2) , since a lot of calculation time 
becomes necessary for the interpolation processing by 
the software, it is difficult to realize high-speed 
analysis - 

SUMMARY OF THE INVENTION 
The present invention has been achieved in order to 
solve the above problems. It is an object of the present 
invention to provide a motion picture analyzing system, 
a motion picture analyzing method and a motion picture 
analyzing program, which can accurately analyze the 
motion of an object in a motion picture at a high speed 
with a simple configuration even when the object is 
moving at a high speed. 

A first aspect of the present invention is a motion 
picture analyzing system that analyzes a motion picture 
consisting of a group of still pictures photographed at 
a discrete time interval, which comprises: an addition 
image generation module that generates a first addition 
image and a second addition image, respectively 
including images of still pictures at different discrete 
time points with the images being added at different 
ratios, by sequentially timewise adding images of still 
pictures at individual discrete time points included in 
the motion picture to be analyzed; and a motion 
detection module that detects a motion component in the 
motion picture to be analyzed, by comparing with each 
other the first addition image and the second addition 
image generated by the addition image generation module. 

In the first aspect of the present invention, it is 
preferable that the addition image generation module 
includes a first memory and a second memory for 
respectively holding the first addition image and the 
second addition image, and the addition image generation 



module generates a new first addition image to be held 
in the first memory by adding a fetched input image and 
the present first addition image held in the first 
memory at a first ratio, while generating a new second 
addition image to be held in the second memory by adding 
the fetched input image and the present second addition 
image held in the second memory at a second ratio 
different from the first ratio. 

In the first aspect of the present invention, it is 
preferable that the motion detection module includes a 
comparator that outputs a motion component by comparing 
with each other, corresponding pixels of the first 
addition image and the second addition image. 

In the first aspect of the present invention, it is 
preferable that the motion picture analyzing system 
further comprises: a first matrix development module 
that takes out a first pixel group located in a 
predetermined area, from the first addition image 
generated by the addition image generation module; and a 
second matrix development module that takes out a second 
pixel group located in an area corresponding to the 
predetermined area, from the second addition image 
generated by the addition image generation module; 
wherein the comparator in the motion detection module 
compares the first pixel group taken out by the first 
matrix development module with the second pixel group 
taken out by the second matrix development module, to 
output a motion component. 

In the first aspect of the present invention, it is 
preferable that the motion detection module includes a 
counter that counts motion components outputted by the 
comparator, to calculate a motion quantity in the motion 
picture to be analyzed. 

In the first aspect of the present invention, it is 
preferable that the motion picture analyzing system 
further comprises: a movement direction detection module 
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that detects a movement direction in the motion picture 
to be analyzed, by comparing the first pixel group taken 
out by the first matrix development module with the 
second pixel group taken out by the second matrix 
5 development module, while spatially shifting these 
groups • 

In the first aspect of the present invention, it is 
preferable that the movement direction detection module 
includes : a motion component output comparator that 

10 outputs a motion component by comparing a first 
reference pixel included in the first pixel group with a 
second reference pixel included in the second pixel 
group; a plurality of direction component output 
comparators that output direction components 

15 corresponding to at least four directions, by 
respectively comparing a plurality of pixels in the 
first pixel group, located at a position shifted in the 
respective directions from the first reference pixel, 
with the second reference pixel in the second pixel 

20 group; and a plurality of mask units that output 
movement direction components corresponding to the 
respective directions, by selectively outputting motion 
components outputted by the motion component output 
comparator, in accordance with the direction components 

25 corresponding to the respective directions outputted by 
the respective direction component output comparators . 

In the first aspect of the present invention, it is 
preferable that the respective mask units output the 
motion component when the direction components 

30 corresponding to the respective directions are larger 
than a predetermined threshold. Moreover, it is 

preferable that the respective mask units output the 
motion component when the direction components 
corresponding to the respective directions and direction 
35 components in a certain relationship therewith are in an 
exclusive relationship . 
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In the first aspect of the present invention, it is 
preferable that the movement direction detection module 
further includes a counter that counts respective 
movement direction components outputted by the 
5 respective mask units to calculate a movement direction 
quantity corresponding to the respective directions. 

A second aspect of the present invention is a 
motion picture analyzing system that analyzes a motion 
picture consisting of a group of still pictures 
10 photographed at a discrete time interval, which 
comprises: an addition image generation module that 
generates a first addition image and a second addition 
image, respectively including images of still pictures 
at different discrete time points with the images being 
15 added at different ratios, by sequentially timewise 
adding images of still pictures at individual discrete 
time points included in the motion picture to be 
analyzed; a first matrix development module that takes 
out a first pixel group located in a predetermined area, 
20 from the first addition image generated by the addition 
image generation module; a second matrix development 
module that takes out a second pixel group located in an 
area corresponding to the predetermined area, from the 
second addition image generated by the addition image 
25 generation module; and a movement direction detection 
module that detects a movement direction in the motion 
picture to be analyzed, by comparing the first pixel 
group taken out by the first matrix development module 
with the second pixel group taken out by the second 
30 matrix development module, while spatially shifting 
these groups . 

In the second aspect of the present invention, it 
is preferable that the addition image generation module 
includes a first memory and a second memory for 
35 respectively holding the first addition image and the 
second addition image, and the addition image generation 
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module generates a new first addition image to be held 
in the first memory by adding a fetched input image and 
the present first addition image held in the first 
memory at a first ratio, while generating a new second 
5 addition image to be held in the second memory by adding 
the fetched input image and the present second addition 
image held in the second memory at a second ratio 
different from the first ratio. 

In the second aspect of the present invention, it 
10 is preferable that the movement direction detection 
module includes: a motion component output comparator 
that outputs a motion component by comparing a first 
reference pixel included in the first pixel group with a 
second reference pixel included in the second pixel 
15 group; a plurality of direction component output 
comparators that output direction components 

corresponding to at least four directions, by 
respectively comparing a plurality of pixels in the 
first pixel group, located at a position shifted in the 
20 respective directions from the first reference pixel, 
with the second reference pixel in the second pixel 
group; and a plurality of mask units that output 
movement direction components corresponding to the 
respective directions, by selectively outputting motion 
25 components outputted by the motion component output 
comparator, in accordance with the direction components 
corresponding to the respective directions outputted by 
the respective direction component output comparators . 

In the second aspect of the present invention, it 
30 is preferable that the respective mask units output the 
motion component when the direction components 
corresponding to the respective directions are larger 
than a predetermined threshold. Moreover, it is 

preferable that the respective mask units output the 
35 motion component when the direction components 
corresponding to the respective directions and direction 
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components in a certain relationship therewith are in an 
exclusive relationship. 

In the second aspect of the present invention, it 
is preferable that the movement direction detection 
5 module further includes a counter that counts respective 
movement direction components outputted by the 
respective mask units to calculate a movement direction 
quantity corresponding to the respective directions. 

A third aspect of the present invention is a 
10 motion picture analyzing method for analyzing a motion 
picture consisting of a group of still pictures 
photographed at a discrete time interval, comprising: a 
step of generating a first addition image and a second 
addition image, respectively including images of still 
15 pictures at different discrete time points with the 
images being added at different ratios, by sequentially 
timewise adding images of still pictures at individual 
discrete time points included in the motion picture to 
be analyzed; and a step of detecting a motion component 
20 in the motion picture to be analyzed, by comparing with 
each other the generated first addition image and the 
generated second addition image. 

In the third aspect of the present invention, it is 
preferable that the method further includes: a step of 
25 taking out a first pixel group located in a 
predetermined area from the generated first addition 
image and taking out a second pixel group located in an 
area corresponding to the predetermined area from the 
second addition image; and a step of detecting a 
30 movement direction in the motion picture to be analyzed, 
by comparing the taken out first pixel group and the 
taken out second pixel group, while spatially shifting 
these groups . 

A fourth aspect of the present invention is a 
35 motion picture analyzing program for analyzing a motion 
picture consisting of a group of still pictures 
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photographed at a discrete time interval, which causes a 
computer to execute: a procedure of generating a first 
addition image and a second addition image, respectively 
including images of still pictures at different discrete 
time points with the images being added at different 
ratios, by sequentially timewise adding images of still 
pictures at individual discrete time points included in 
the motion picture to be analyzed; and a procedure of 
detecting a motion component in the motion picture to be 
analyzed, by comparing the generated first addition 
image and the generated second addition image. 

In the fourth aspect of the present invention, it 
is preferable that the program further causes a computer 
to execute: a procedure of taking out a first pixel 
group located in a predetermined area from the generated 
first addition image and taking out a second pixel group 
located in an area corresponding to the predetermined 
area from the second addition image; and a procedure of 
detecting a movement direction in the motion picture to 
be analyzed, by comparing the taken out first pixel 
group and the taken out second pixel group, while 
spatially shifting these groups. 

According to the present invention, after two kinds 
of images (a first addition image and a second addition 
image respectively including images of still pictures at 
different discrete time points at different ratios) are 
generated by sequentially timewise adding images of 
still pictures at individual discrete time points 
included in a motion picture to be analyzed, a motion 
component in the motion picture to be analyzed is 
detected by comparing the two kinds of images generated 
in this manner. Here, since the two kinds of images to 
be compared include events occurring at different 
discrete time points, the result obtained by comparing 
the first addition image with the second addition image 
is not discrete, even if the photographing interval of 



the still pictures is discrete. Therefore, even when the 
movement of an object to be photographed is fast, the 
motion components in the motion picture can accurately 
be analyzed at a high speed with a simple configuration. 
Particularly, a circuit which realizes such processing 
can easily be realized by hardware, and the two kinds of 
images to be compared are obtained by temporally adding 
a plurality of images. Therefore, the noiseproof 
property can be improved by the temporal averaging 
effect, without sacrificing time resolution. 

Moreover, according to the present invention, the 
movement direction in the motion picture to be analyzed 
is detected by taking out the first pixel group located 
in a predetermined area from the generated first 
addition image, as well as taking out the second pixel 
group located in an area corresponding to the 
predetermined area from the second addition image, and 
then comparing the taken out first pixel group and the 
taken out second pixel group, while spatially shifting 
these groups. As a result, the movement direction in the 
motion picture can be analyzed with a simple 
configuration . 

BRIEF DESCRIPTION OF THE DRAWINGS 

By referring to the accompanying drawings, 
embodiments of the present invention will be described 
hereinafter. In the drawings, 

Fig. 1 is a block diagram illustrating the overall 
configuration of a motion picture analyzing system 
according to an embodiment of the present invention; 

Fig. 2 is a block diagram illustrating an example 
of an addition image generation module in the motion 
picture analyzing system shown in Fig. 1; 

Fig. 3 is a conceptual diagram illustrating an 
example of addition images (image A and image B) 
generated by the addition image generation module in the 
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motion picture analyzing system shown in Fig. 1; 

Fig. 4 is a block diagram illustrating another 
example of the addition image generation module in the 
motion picture analyzing system shown in Fig. 1; 
5 Fig. 5 is a block diagram illustrating an example 

of a matrix development module in the motion picture 
analyzing system shown in Fig. 1; 

Fig. 6 is a block diagram illustrating an example 
of a comparator included in the motion detection module 
10 in the motion picture analyzing system shown in Fig. 1; 

Fig. 7 is a block diagram illustrating another 
example of the comparator included in the motion 
detection module in the motion picture analyzing system 
shown in Fig . 1 ; 
15 Fig. 8 is a block diagram illustrating an example 

of a mask unit included in the motion detection module 
in the motion picture analyzing system shown in Fig. 1; 

Fig. 9 is a block diagram illustrating another 
example of a mask unit included in the motion detection 
20 module in the motion picture analyzing system shown in 
Fig. 1; and 

Fig. 10 is a conceptual diagram for explaining the 
function of the mask unit shown in Figs. 8 and 9. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
At first , the overall configuration of the motion 
picture analyzing system according to an embodiment of 
the present invention will be described, with reference 
to Fig. 1. 

As shown in Fig. 1, the motion picture analyzing 
system 1 includes an addition image generation module 10, 
matrix development modules 21 and 22, a motion detection 
module 30, and a movement direction detection module 40. 

Among the above modules, the addition image 
generation module 10 is for generating two kinds of 
addition images (image A and image B) . The two kinds of 
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addition images respectively includes images of still 
pictures at different discrete time points with the 
images being added at different ratios. The images 
(input images) of still pictures are sequentially 
5 timewise added at individual discrete time points 
included in the motion picture to be analyzed. 

The matrix development modules 21 and 22 are for 
respectively taking out pixel groups (first pixel group 
a(i, j) and second pixel group b(i, j)) located in a 
10 predetermined region (for example, a region having a 
size of 3 x 3) , from the image A and the image B 
generated by the addition image generation module 10. 

The motion detection module 30 is for detecting a 
motion component in the motion picture to be analyzed, 
15 by comparing with each other the first pixel group a(i, 
j) and the second pixel group b(i, j), respectively 
taken out by the matrix development modules 21 and 22. 

The movement direction detection module 4 0 is for 
detecting a movement direction in the motion picture to 
20 be analyzed, by comparing with each other the first 
pixel group a(i, j) and the second pixel group b(i, j), 
respectively taken out by the matrix development modules 
21 and 22, while spatially shifting these groups. 

The addition image generation module 10, the matrix 
25 development modules 21 and 22, the motion detection 
module 30 and the movement direction detection module 4 0 
shown in Fig. 1 will be described in detail, with 
reference to Figs. 2 to 10. 
<Addition image generation module> 
30 At first, the details of the addition image 

generation module 10 will be described with reference to 
Fig. 2. 

As shown in Fig . 2 , the addition image generation 
module 10 includes two memories 14 and 18 respectively 
35 having a capacity capable of holding an image for one 
still picture. 
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The addition image generation module 10 includes an 
analog multiplier 11 which multiplies an input image by 
(1/n) , an analog multiplier 12 which multiplies an image 
held in the memory 14 by (1/m) , and an adder 13 which 
adds the results outputted from the analog multipliers 
11 and 12. As a result, in the addition image generation 
module 10 , the fetched input image and the present image 
held in the memory 14 are added at a first ratio ((1/n) 
to (1/m)), and the new image obtained in this manner is 
stored again in the same address in the memory 14, and 
outputted as an image A. The initial value of the memory 
14 is preferably an input * image immediately before 
starting the addition, or an optional fixed value. 

The addition image generation module 10 also 
includes an analog multiplier 15 which multiplies an 
input image by (1/n'), an analog multiplier 16 which 
multiplies an image held in the memory 18 by (l/m T )/ and 
an adder 17 which adds the results outputted from the 
analog multipliers 16 and 17. As a result, in the 
addition image generation module 10, the fetched input 
image and the present image held in the memory 18 are 
added at a second ratio ((l/n f ) to (1/m')), and the new 
image obtained in this manner is stored again in the 
same address in the memory 18, and outputted as an image 
B. The initial value of the memory 18 is preferably an 
input image immediately before starting the addition, or 
an optional fixed value. 

The first ratio and the second ratio described 
above need to satisfy a relation of "1/n + 1/m = 1," and 
M l/n f + 1/m 1 = 1." Moreover, it is necessary that the 
first ratio and the second ratio are different from each 
other, for example, the ratio for the image A is (1/2) 
to (1/2) , and the ratio for the image B is (1/4) to 
(3/4) . 

As a result, the image A and the image B held in 
the memories 14 and 18 are obtained by adding images in 
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still pictures at different discrete time points, and 
when n < n', the image A includes more events close to 
the present than the image B. In other words, due to the 
difference in ratio at which the input image and the 
5 present image held in the memories 14 and 18 are added, 
one (for example, the image A) of the images A and B is 
an image reflecting the latest event, and the other (for 
example, the image B) is an image reflecting the former 
event. Therefore, by comparing these images A and B, the 
10 motion component of the motion picture can be detected. 

Fig. 3 is a conceptual diagram illustrating one 
example of the image A and the image B obtained in this 
manner. In Fig. 3, both of the image A and the image B 
are images obtained when an elliptical object having 
15 brightness constant timewise moves from the left side to 
the right side (in Fig. 3, as the object becomes 
brighter, the color thereof is expressed darker) . In the 
image A and the image B, the first ratio and the second 
ratio are in the relation of n < n ' , and the image A 
20 includes more events close to the present than the image 
B. 

In the addition image generation module 10 shown in 
Fig. 2, the input image is multiplied by (1/n) or (1/n 1 ), 
and the images held in the memories 14 and 18 are 

25 respectively multiplied by (1/m) or (1/m 1 ), and then the 
both values are added. However, as in an addition image 
generation module 60 shown in Fig. 4, the input image is 
multiplied by n or n f , and the images held in the 
memories 14 and 18 are respectively multiplied by m or 

30 m f , and then the both values may be added, and 
multiplied by (l/(n+m)) or (l/(n f +m')) after the 
addition . 

In other words, as shown in Fig. 4, the addition 
image generation module 60 includes: a memory 65 having 
35 a capacity capable of holding an image for one still 
picture; an analog multiplier 61 which multiplies an 
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input image by n; an analog multiplier 62 which 
multiplies an image held in the memory 65 by m; an adder 
63 which adds the results outputted from the analog 
multipliers 61 and 62; and an analog multiplier 64 which 
5 multiplies the result outputted from the adder 63 by 
(l/(n+m)). As a result, in the addition image generation 
module 60, the fetched input image and the present image 
held in the memory 65 are added at a first ratio ((1/n) 
to (1/m)), and the new image obtained in this manner is 
10 stored again in the same address in the memory 65, and 
outputted as an image A. 

Moreover, the addition image generation module 60 
includes: a memory 70 having a capacity capable of 
holding an image for one still picture; an analog 
15 multiplier 66 which multiplies an input image by n"; an 
analog multiplier 67 which multiplies an image held in 
the memory 70 by m' ; an adder 68 which adds the results 
outputted from the analog multipliers 66 and 67; and an 
analog multiplier 69 which multiplies the result 
20 outputted from the adder 68 by <l/(n'+m')). As a result, 
in the addition image generation module 60, the fetched 
input image and the present image held in the memory 70 
are added at a second ratio ((1/n') to (1/m')), and the 
new image obtained in this manner is stored again in the 
25 same address in the memory 70, and outputted as an image 
B. 

In the addition image generation module 60 shown in 
Fig. 4, it is necessary to take a sufficient bit width 
in the adders 63 and 68; but since there is no problem 
30 of bit dropouts due to performing division in the 
initial stage, more precise result can be outputted. 

Here, the characteristics of the image A and the 
image B obtained by adding the input image and the 
present image will be described in more general terms, 

35 as follows . 

If the information of a pixel at a time point t at 
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the pixel position (x, y) in the image is expressed as Y 0 
(x, y , t) , the image to be held in the memory conforms 
to a recurrence formula in the following formula (1) : 

Y(x,y^ = ^S^xY 0 {x,y,n)+\l-^yY{x,y,n-k)^ ^ - 0) 

wherein n is a discrete value which changes from k to t 
by 1, that is, n = k, k+1, k+2 , ... , t, and corresponds to 
the photographing frame number; and k is a positive real 
number, and represents an interval between the 
photographing frames. Moreover, r is a positive real 
number, and represents an addition ratio. Moreover, an 
image Y (x, y, 0) at the photographing frame number n = 
k is an initial value, and is an input image immediately 
before starting the addition, or an optional fixed value. 

According to the above formula (1) , if the two 
kinds of images (average images) are expressed as Y f (x, 
y, t) and Y s (x, y, t) , the images to be held in the 
memories become as shown in the following formulas (2) 
and (3) : 

- (2) 



1 i ( 1 ^ 

YAx,y,t)=^\ — xY 0 (x,y,n)+ 1-— *Y f (x,y,n-k,) 
1 tt.\r r \ r f I 



V r fJ 



and 



Y/x,y l t)=X\-xY 0 (x,y,n) + 



\-±)xY.fcy.n-k,)\ - (3) 



n-k s \J S 



Generally, either of r f * r s and k f ^ k s should be 
held true, or both may be held true. For example, if r f 
< r s and k f = k s , Y f (x, y, t) includes more events close 
to the present than Y s (x, y, t) . Likewise, when r f = r s 
and k f < k s , the same thing applies. 
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Therefore, by comparing these two kinds of images, 
the motion component in the motion picture can easily be 
detected. 

Moreover, by comparing these two kinds of images, 
5 while spatially shifting these groups, that is, by 
comparing Y f (x+Ax, y+Ay, t) with Y s (x, y, t) , the 
movement direction in the motion picture can be detected. 

In the above description, in the addition image 
generation modules 10 and 60, the two kinds of images 

10 (the image A and the image B) include images of the 
still pictures at different discrete time points at 
different ratios, by changing the ratio of adding the 
input image to the present images respectively held in 
the memories 14 and 18 and the memories 65 and 70. 

15 However, not being limited thereto, the two kinds of 
images (the image A and the image B) may include images 
of the still pictures at different discrete time points 
at different ratios, by changing a frame in which the 
input image is added to the present images respectively 

20 held in the memories 14 and 18 and the memories 65 and 
70 . The latter case corresponds to k * 0 in the above 
formula (1), and k f * 0 and k s * 0 in the formulas (2) 
and (3) . 

<Matrix development module> 

25 The details of the matrix development modules 21 

and 22 will be described with reference to Fig. 5. Here, 
an example in which a pixel group located in an area 
(window area) having a size of 3 x 3 is taken out from 
the image will be described. 

30 As shown in Fig. 5, the matrix development modules 

21 and 22 respectively include line memories 23 and 24 
in the horizontal direction, and eleven memories 25, 26 
and 27 for holding the information for one pixel. In 
such matrix development modules 21 and 22, the 

35 information inputted from the addition image generation 
module 10 in a unit of pixel is sequentially stored in 
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the line memories 23 and 24 and the memories 25 and 26, 
and sequentially outputted to the 9x9 memories (pixel 
memories) 27 by line. As a result, the information of 3 
x 3 pixels in an arbitrary area in the image is stored in 
the nine pixel memories at all times, and is outputted 
as 3 x 3 matrices 0(i, j) (the first pixel group a(i, j) 
or the second pixel group b(i, j)) . 

The first pixel group a(i, j) and the second pixel 
group b(i, j) taken out by the matrix development 
modules 21 and 22 are inputted to the motion detection 
module 30 and the movement direction detection module 40. 
<Movement direction detection module> 

As shown in Fig. 1, the motion detection module 30 
includes a comparator 31 which compares with each other, 
corresponding pixels of the image A and the image B and 
outputs the motion component. 

As the comparator 31, as shown in Fig. 6, a 
comparator which compares arbitrary pixels of the first 
pixel group a(i, j) and the second pixel group b(i, j) 
taken out by the matrix development modules 21 and 22 
can be used. 

As shown in Fig. 6, the comparator 31 includes a 
subtracter 33 which takes a difference between the input 
scalar quantities a and b, and an absolute value unit 34 
which takes an absolute value in the result outputted 
from the subtracter 33. As a result, in the comparator 
31, a difference in the pixel information inputted as 
the scalar quantities a and b (for example, a (2, 2) and 
b(2, 2) located at the center of the first pixel group 
a(i, j) and the second pixel group b(i, j)) is taken, 
and the absolute value is outputted as the comparison 
result. 

The comparison result outputted from the comparator 
31 represents a temporal change (motion component) of 
corresponding pixels of the image A and the image B. 
Therefore, an image obtained by extracting only the 



18 



motion component in the motion picture (a motion image) 
can be obtained based on the comparison result. 

In the comparator 31 shown in Fig. 6, the 
comparison result is output ted based on the optional 
pixel information (for example, a (2, 2) and b(2, 2) 
located at the center of the pixel groups) . However, the 
comparison result may be outputted based on the whole 
pixel information of the first pixel group a(i, j) and 
the second pixel group b(i, j) located in 3 x 3 window 
areas . 

In other words, as shown in Fig. 7, the comparator 
35 includes nine scalar quantity comparators 36 which 
take a difference between corresponding elements 
(elements with the same subscript) included in the input 
3x3 matrix quantities a(i, j) and b(i, j) , and an adder 
37 which takes the sum of the results outputted from the 
scalar quantity comparators 36. The comparator 31 shown 
in Fig. 6 can be used for the scalar quantity comparator 
36. As a result, the comparator 35 takes a difference 
between nine elements (a(i, j) and b(i, j)) having the 
same subscript, and outputs the sum of the absolute 
values as the comparison result. The comparator 35 shown 
in Fig. 7 simply calculates a difference between a(i, j) 
and b(i, j), but correlation between a(i, j) and b(i, j) 
may be calculated by an existing optional method. 

The motion detection module 30 is provided with a 
counter 32 which counts the output result (motion 
component) outputted from the comparator 31 . By adding 
up a temporal change (motion component) of each pixel 
over a part or all of the image by the counter 32, a 
local or overall motion quantity in the motion picture 
can be obtained. 

<Movement direction detection module> 

As shown in Fig. 1, the movement direction 
detection module 4 0 includes a motion component output 
comparator 41a that compares a (2, 2) located at the 
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center of the first pixel group a(i, j) with b(2,2) 
located at the center of the second pixel group b(i, j) 
to output a motion component (M) , and a plurality of 
direction component output comparators 41b that 
5 respectively compares a plurality of pixels (a (2, 3), 
ad, 2), a (2, 1), a(3, 2)) located at a position shifted 
in four directions (+x, +y, -x, -y) from a (2, 2) of the 
first group a(i, j) with b(2,2) in the second pixel 
group b(i, j) to output direction components (D) 
10 corresponding to the respective directions. 

The movement direction detection module 40 also 
includes a plurality of mask units 42 that selectively 
outputs the motion component (M) outputted by the motion 
component output comparator 41a in accordance with the 
15 direction components (D) corresponding to the respective 
directions outputted by the respective direction 
component output comparators 41b, thereby outputting 
movement direction components corresponding to the 
respective directions (+x, +y, -x, -y) . 
20 Fig. 8 is a diagram showing such mask units 42. As 

shown in Fig. 8, the respective mask units 42 include a 
magnitude comparator 44 that compares the magnitudes of 
the direction components (D) and a threshold (T) , and 
when D > T, outputs "1", and when D < T, outputs "0". The 
25 respective mask units 42 also include an AND circuit 45 
that takes a logical product of the motion component (M) 
and the output result of the magnitude comparator 44. As 
a result, the respective mask units 42 output the motion 
component (M) when the direction components (D) 
30 corresponding to the respective directions are larger 
than the predetermined threshold (T) , and output n 0" 
otherwise. The results outputted from the respective 
mask units 42 represent a motion component (movement 
direction component) of a pixel moving only in one 
35 direction of <+x, +y, -x, -y) . Therefore, based on the 
results, an image that extracts a motion component of a 
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motion picture moving only in a specific direction (a 
movement direction image) can be obtained. 

The mask units 42 shown in Fig. 8 selectively 
output the motion component (M) based on only one 
direction component (D) . However, the mask units 42 may 
selectively output the motion component (M) based on 
more than two direction components (D, D 1 ) , as shown in 
Fig. 9. 

In other words, as shown in Fig. 9, the respective 
mask units 4 6 include a magnitude comparator 47 that 
compares the magnitude of the direction components (D) 
and a threshold (T) , and when D > T, outputs "1", and 
when D < T, outputs "0". The respective mask units 46 
also include a magnitude comparator 48 that compares the 
magnitude of the direction components (D f ) in a 
direction shifted by 90 degrees from the direction of 
the direction components (D) with the threshold (T) , and 
when D 1 < T , outputs "1", and when D' > T, outputs "0". 
The respective mask unit 4 6 further include an AND 
circuit 4 9 that takes a logical product of the output 
result of the magnitude comparator 47 and the output 
result of the magnitude comparator 48, and an AND 
circuit 50 that takes a logical product of the motion 
component (M) and the output result of the AND circuit 
49. As a result, the respective mask units 46 output the 
motion component (M) when the direction component (D) 
corresponding to the respective directions (for example, 
+x) and the direction component (D 1 ) in a certain 
relationship therewith (a direction shifted by 90 
degrees (for example, +y) are in an exclusive 
relationship, and output "0" otherwise. 

Here, explanation is given to a case where a motion 
component (movement direction component) of a pixel 
moving only in a specific direction (for example, +x 
direction) is extracted. In the mask units 4 4 shown in 
Fig. 8, as shown in Fig. 10(a), a portion (shaded 
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portion) corresponding to a difference between an image 
A f (an image obtained by shifting the image A by Ax) and 
the image B is designated as a selective output portion 
(mask corresponding portion) , and only a motion 
5 component (M) in this portion is selectively outputted. 
On the other hand, in the mask unit 46 shown in Fig. 9, 
as shown in Fig. 10(b), a portion (cross-hatching 
portion) obtained by subtracting a portion (dot portion) 
corresponding to a difference between an image Ai" or an 
10 image A 2 " (an image obtained by shifting the image A by 
Ay or -Ay) and the image B, from a portion (shaded 
portion) corresponding to a difference between the image 
A f (an image obtained by shifting the image A by Ax) and 
the image B is designated as a selective output portion 
15 (mask corresponding portion) , and only a motion 
component (M) in this portion is selectively outputted. 

Generally, when a diagonal direction component is 
included in the shape (external shape) of an object, 
even when the object is actually moving in the +x 
20 direction, a motion component in the ±y direction appears. 
In other words, in Fig. 10(b), in a portion (see 
reference symbols 81 and 82) overlapping on a portion 
(dot portion) corresponding to a difference between the 
image Ax" (or an image A 2 ") obtained by shifting the 
25 image A by Ay (or -Ay) and the image B, of the portion 
(shaded portion) corresponding to a difference between 
the image A' (an image obtained by shifting the image A 
by Ax) and the image B, even when the object moves in 
the +x direction or in the ±y direction, the motion 
30 component (M) appears. In the mask unit 46 shown in Fig. 
9, however, since these portions can be excluded from 
the selective output portion (mask corresponding 
portion) , a motion component (movement direction 
component) of the pixel moving only in a specific 
35 direction (for example, +x direction) can accurately and 
reliably be extracted, regardless of the object shape. 
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In the movement direction detection module 40, a 
plurality of counters 43 that counts the output results 
(movement direction components) outputted by the 
respective mask units 42 for each direction (+x, +y, -x, 
-y) is provided. By adding up the movement direction 
components of the respective pixels for each direction 
(+x, +y, -x, -y) over a part or all of the image by the 
counters 43, a local or overall movement direction 
quantity in the motion picture corresponding to each 
direction (+x, +y, -x, -y) can be obtained. 

Here, by using the movement direction quantities in 
four directions (+x, +y, -x, -y) , the movement direction 
in the motion picture can be caught in an optional 
direction. In other words, when either one of the +x 
movement direction quantity and the -x movement 
direction quantity, having a larger absolute value, is 
designated as X, and either one of the +y movement 
direction quantity and the -y movement direction 
quantity, having a larger absolute value, is designated 
as Y, to create a vector (X, Y) , the angle 0 
representing the movement direction in the motion 
picture is expressed as: 
0 = tan" 1 (Y/X) . 

According to the embodiment, after two kinds of 
images (the image A and the image B respectively 
including images of still pictures at different discrete 
times at different ratios) are generated by sequentially 
timewise adding images of still pictures at individual 
discrete time points included in a motion picture to be 
analyzed by the addition image generation module 10, a 
motion component in the motion picture to be analyzed is 
detected by comparing the two kinds of images (the image 
A and the image B) generated in this manner by the 
motion detection module 30. Here, since the two kinds of 
images (the image A and the image B) to be compared 
include events occurring at different discrete time 
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points, the result obtained by comparing the image A 
with the image B is not discrete, even if the 
photographing interval of the still pictures is discrete. 
Therefore, even when the movement of an object to be 
photographed is fast, the motion components in the 
motion picture can accurately be analyzed at a high 
speed with a simple configuration. Particularly, a 
circuit which realizes such processing can easily be 
realized by hardware, and the two kinds of images (the 
image A and the image B) to be compared are obtained by 
temporally adding a plurality of images . Therefore, the 
noiseproof property can be improved by the temporal 
averaging effect, without sacrificing time resolution. 

Moreover, according to the embodiment, after the 
first pixel group a(i, j) located in a 3 x 3 window area 
is taken out from the image A generated by the addition 
image generation module 10, and the second pixel group 
b(i, j) located in an area corresponding to the window 
area is taken out from the image B by the addition image 
generation module 10, by using the matrix development 
modules 21 and 22, the taken out first pixel group a(i, 
j) and the taken out second pixel group b(i, j) are 
compared by the movement direction detection module 40, 
while spatially shifting these groups, thereby detecting 
a movement direction in the motion picture to be 
analyzed. As a result, the movement direction in the 
motion picture can be analyzed with a simple 
configuration . 

The movement direction in the motion picture 
obtained in this manner can be used in various kinds of 
control systems such as a robot. 

In the embodiment, the comparator 31 in the motion 
detection module 30 compares with each other the first 
pixel group a(i, j) and the second pixel group b(i, j) 
taken out by the matrix development modules 21 and 22 to 
detect the motion component in the motion picture to be 
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analyzed. However, when optional pixels in the first 
pixel group a(i, j) and the second pixel group b(i, j) 
are compared, the corresponding pixels in the image A 
and the image B outputted from the addition image 
generation module 10 may be directly compared, without 
using the matrix development modules 21 and 22. 

In the embodiment, in the movement direction 
detection module 40, the direction components (D) 
obtained by taking an absolute value of a difference 
between the spatially shifted image A and image B is 
used in the respective mask units 42. However, not 
limited thereto, one obtained by appropriately 
calculating the direction components (D) may be used in 
the respective mask units 42. 

Moreover , in the embodiment , in the movement 
direction detection module 40, the motion component 
output comparator 41a and the mask units 42 are used in 
addition to the respective direction component output 
comparators 41b as the means for selectively outputting 
the motion component (M) . However, if the movement 
direction quantity and the angle 0 representing only the 
movement need to be obtained, only the direction 
component output comparators 41b that output the 
direction components (D) corresponding to the respective 
directions are necessary. In this case, the motion 
component output comparator 41a and the mask units 42 
are not necessary. 

In the embodiment, the addition image generation 
module 10, the matrix development modules 21 and 22, the 
motion detection module 30 and the movement direction 
detection module 40 can be realized as a program 
operated on a computer. Such a program is stored in a 
computer readable recording medium such as a memory on a 
computer, a hard disk, a flexible disk, or a CD-ROM, and 
sequentially read out from the CPU (Central Processing 
Unit) on the computer and executed, to realize the 



above-described functions 



